Object identification method, apparatus and device

ABSTRACT

Provided are an object identification method, apparatus, and device. The object identification method comprises: acquiring a first image of at least part of an object; determining a feature portion of the object on the basis of the first image; acquiring a second image of the feature portion of the object; and identifying an object category of the object on the basis of the second image. According to the object identification method, apparatus, and device, in which a feature portion of an object is acquired and identification of the category of the object is performed on the basis of the feature portion, operations are simple, and the accuracy of object identification can be effectively improved.

BACKGROUND Technical Field

The present disclosure relates to the field of computer technologies,and in particular, to an object identification method, apparatus, anddevice.

Description of Related Art

The field of computer technology sees the need for identifying a varietyof objects. However, in most cases, users need to search for andidentify the category of the object by themselves through searchengines, terminology dictionaries and other assistance tools accordingto the features of the object. Conducting search in this way istime-consuming and the results are not very accurate. In recent years,there have been applications in which category information of the objectis obtained by taking an image of the object and then using the image asan input. However, due to the problems such as the scope of imagescaptured by users might be too broad, lack of detailed information, notincluding key feature parts of objects and so on, object categoryidentification that is performed directly based on captured images mightlead to inaccurate results. Accordingly, there is a need for improvingmethods and apparatus for identifying category of objects.

SUMMARY

The disclosure provides an object identification method, which includes:acquiring a first image of at least part of the object; determining afeature portion of the object based on the first image; acquiring asecond image of the feature portion of the object; and identifying anobject category of the object based on the second image.

In an embodiment of the present disclosure, the step of determining thefeature portion of the object includes: identifying a preliminarycategory of the object based on the first image; and determining thefeature portion of the object based on the identified preliminarycategory of the object.

In an embodiment of the present disclosure, the step of obtaining thesecond image of the feature portion of the object includes: providingprompt information to the user, the prompt information instructing theuser to input the second image of the feature portion of the object; andreceiving the second image of the feature portion of the object.

In an embodiment of the present disclosure, the step of acquiring thesecond image of the feature portion of the object includes: determiningwhether the first image includes the feature portion of the object; andwhen the first image includes the feature portion of the object:cropping the first image to obtain the image of the feature portion asthe second image, or in the case where the first image is areduced-resolution image after down-sampling the third image, croppingthe third image to obtain the image of the feature portion as the secondimage.

In an embodiment of the present disclosure, the step of determiningwhether the first image includes the feature portion of the objectincludes: identifying and labeling each part of the object in the firstimage through a pre-trained first object part identification model; anddetermining whether the first image includes the feature portion of theobject based on the identification and labeling results.

In an embodiment of the present disclosure, the step of acquiring asecond image of the feature portion of the object includes: determiningwhether the first image includes a complete image of the object; and inthe case that the first image includes the complete image of the object:cropping the first image to obtain the image of the feature portion asthe second image, or in the case where the first image is areduced-resolution image after down-sampling the third image, croppingthe third image to obtain the image of the feature portion as the secondimage.

In an embodiment of the present disclosure, the object is a plant, andthe resolution of the second image is higher than that of the firstimage. The step of identifying the preliminary category of the objectincludes: acquiring and recording the one or more of locationinformation and season information in acquisition of the first image,excluding impossible object categories based on the one or more oflocation information and season information; and in the case ofexcluding impossible object categories, identifying the preliminarycategory of the object.

In an embodiment of the present disclosure, the step of identifying theobject category of the object includes identifying one or more ofcategory information, location information, season information, timeinformation, weather information, and capturing angle informationassociated with the object.

In an embodiment of the present disclosure, one or more of the firstimage and the second image are stored in a sample library correspondingto the object category of the object, and physiological cycleinformation and appearance information corresponding to the one or moreof location information, season information, time information, weatherinformation, and capturing angle information are recorded.

In an embodiment of the present disclosure, the step of providing theprompt information to the user includes: providing the promptinformation to the user through one or more of text, graphics, andvoice.

In an embodiment of the present disclosure, the preliminary category ofthe object is identified based on a pre-trained first object categoryidentification model. The object category of the object is identifiedbased on a pre-trained second object category identification model. Thefirst object category identification model and the second objectcategory identification model are the same or different. The objectcategory identification model includes a deep convolutional neuralnetwork or a deep residual network.

In an embodiment of the present disclosure, the training step of thefirst and/or object category identification model includes: acquiring atraining sample set, each sample in the training sample set is labeledwith a corresponding category; acquiring a test sample set, each samplein the test sample set is labeled with a corresponding category, and thetest sample set is different from the training sample set; training theobject category identification model based on the training sample set;testing the object category identification model based on the testsample set; when the test result indicates that the identificationaccuracy rate of the object category identification model is less than apreset accuracy rate, increasing the number of samples in the trainingsample set for retraining; and when the test result indicates that theidentification accuracy rate of the object category identification modelis greater than or equal to the preset accuracy rate, completing thetraining.

The present disclosure provides an apparatus for object identification,which includes: an image acquisition module configured to acquire afirst image of at least a portion of the object, and to acquire a secondimage of a feature portion of the object; a feature portion determiningmodule configured to determine a feature portion of the object based onthe first image; and an object category identification module configuredto identify an object category of the object based on the second imageof the feature portion of the object.

In an embodiment of the present disclosure, the step of determining thefeature portion of the object includes: identifying a preliminarycategory of the object based on the first image; and determining thefeature portion of the object based on the identified preliminarycategory of the object.

In an embodiment of the present disclosure, the step of obtaining thesecond image of the feature portion of the object includes: providingprompt information to the user, the prompt information instructing theuser to input the second image of the feature portion of the object; andreceiving the second image of the feature portion of the object.

In an embodiment of the present disclosure, the step of acquiring thesecond image of the feature portion of the object includes: determiningwhether the first image includes the feature portion of the object; andwhen the first image includes the feature portion of the object:cropping the first image to obtain the image of the feature portion asthe second image, or in the case where the first image is areduced-resolution image after down-sampling the third image, croppingthe third image to obtain the image of the feature portion as the secondimage.

In an embodiment of the present disclosure, the step of acquiring asecond image of the feature portion of the object includes: determiningwhether the first image includes a complete image of the object; and inthe case that the first image includes the complete image of the object:cropping the first image to obtain the image of the feature portion asthe second image, or in the case where the first image is areduced-resolution image after down-sampling the third image, croppingthe third image to obtain the image of the feature portion as the secondimage.

The present disclosure provides a device for object identification,which includes: an image acquisition unit for acquiring an input image;a processor; and a memory configured to store a series ofcomputer-executable instructions and a series of computer-accessibledata associated with the series of computer-executable instructions, andthe series of computer-executable instructions enable the processor toperform the method described in any embodiment in the present disclosurewhen being executed by the processor.

The present disclosure provides a computer-readable storage mediumhaving computer-executable instructions stored therein, and thecomputer-executable instructions enable the processor to perform themethod described in any embodiment in the present disclosure when beingexecuted by the processor.

In the object identification method, apparatus and device provided bythe disclosure, by acquiring the feature portion of the object andidentifying the category of the object based on the feature portion, theoperation is simple and the accuracy of the object identification may beeffectively improved.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to illustrate the technical solutions of the embodiments of thepresent disclosure more clearly, the accompanying drawings of theembodiments will be briefly introduced below. Clearly, the drawings inthe following description only relate to some embodiments of the presentdisclosure, rather than limit the present disclosure.

FIG. 1 shows a schematic view of a network environment of an objectidentification system according to an embodiment of the presentdisclosure.

FIG. 2 shows a flowchart of an object identification method according toan embodiment of the present disclosure.

FIG. 3 shows a flowchart of a training method for an object categoryidentification model according to an embodiment of the presentdisclosure.

FIG. 4 shows a flowchart of an object identification method according toanother embodiment of the present disclosure.

FIG. 5 shows a schematic view of an object identification apparatusaccording to an embodiment of the present disclosure.

FIG. 6 shows a schematic view of an object identification deviceaccording to an embodiment of the present disclosure.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments of the present disclosure will bedescribed in detail below with reference to the accompanying drawings.It should be noted that the relative arrangement of the components andsteps, the numerical expressions and numerical values set forth in theseembodiments do not limit the scope of the present disclosure unlessspecifically stated otherwise. In the following description, in order tobetter explain the present disclosure, numerous details are set forth,however it should be understood that the present disclosure may beimplemented without these details.

The following description of various exemplary embodiments is merelyillustrative, and those of ordinary skill in the art will understandthat other variations, modifications, and alternatives are possible. Inthe present disclosure, the terms “first”, “second”, etc. are only usedto distinguish between elements or steps, etc., and are not intended todenote chronological order, priority, or importance.

Techniques, methods, and apparatus known to those of ordinary skill inthe art may not be discussed in detail herein, but where appropriate,such techniques, methods, and apparatus should be considered part ofthis specification.

The inventors of the present disclosure have intensively studied methodsand systems for object identification. In order to simplify thedescription, plants are used as examples of objects in the followingexemplary embodiments, but it should be construed that “objects” in thepresent disclosure include but are not limited to animals, people,scenery, natural objects, buildings, commodities, food, medicines,and/or daily necessities, etc.

FIG. 1 shows a schematic view of a network environment 100 of an objectidentification system according to an embodiment of the presentdisclosure.

The network environment 100 of the object identification system mayinclude a mobile device 102, a remote server 103, a training device 104and a database 105, which are wired or wirelessly coupled to each otherthrough a network 106. The network 106 may be embodied as a wide areanetwork (such as a mobile telephone network, public switched telephonenetwork, satellite network, the Internet, etc.), a local area network(such as Wi-Fi, Wi-Max, ZigBee™ Bluetooth™, etc.), and/or other forms ofnetworking capabilities.

The mobile device 102 may include mobile phones, tablet computers,laptop computers, personal digital assistants, and/or other computingdevices configured to capture, store, and/or transmit images such asdigital photographs. Accordingly, the mobile device 102 may include animage capturing device, such as a digital camera, and/or may beconfigured to receive images from other devices. The mobile device 102may include a display. The display may be configured to provide the user101 with one or more user interfaces, which may include a plurality ofinterface elements, and the user 101 may interact with the interfaceelements, and the like. For example, the user 101 may use the mobiledevice 102 to capture an object and upload or store the image of theobject. The mobile device 102 may output category information anddetailed description about the object to the user, or may output promptinformation and the like to instruct the user to capture a specific partof the object.

The remote server 103 may be configured to analyze an image of theobject or the like received from the mobile device 102 through thenetwork 106 to determine the category of the object, and to provideinformation such as a detailed description of the corresponding object.The remote server 103 may further be configured to create and train anobject category identification model according to an embodiment of thepresent disclosure. The specific training process of the object categoryidentification model will be described below in conjunction withspecific embodiments.

The training device 104 may be coupled to the network 106 to facilitatetraining of the object category identification model. The trainingdevice 104 may have multiple CPUs and/or Graphic Processing Units (GPUs)to assist in training the object category identification model.

The database 105 may be coupled to the network 106 and provide the dataneeded by the remote server 103 to perform related calculations. Forexample, the database 105 may include a sample library that storesimages of a large number of objects of different categories. In anembodiment, taking plants as an example, the sample library may includea large number of image samples of different categories of plants indifferent locations, different seasons, different weathers at differenttimes, and different capturing angles. In an embodiment, the selectedplant photos captured by the user may also be stored in a sample librarycorresponding to the plant category. Meanwhile, physiological cycleinformation and appearance information corresponding to one or more ofthe location information, season information, time information, weatherinformation and capturing angle information of the plant may also berecorded in the database. The database may be implemented using variousdatabase techniques known in the art. The remote server 103 may accessthe database 105 for relevant operations as needed.

It should be understood that the network environment 100 herein ismerely an example. Those skilled in the art may add more devices ordelete some devices as needed, and may modify the functions andconfigurations of some devices. In the following, description will begiven by taking the object to be identified as a plant as an example.

The object identification method 200 according to an embodiment of thepresent disclosure is described below with reference to FIG. 2 .

FIG. 2 shows a flowchart of an object identification method 200according to an embodiment of the present disclosure. As shown in FIG. 2, in step S201, a first image of at least a part of the object isacquired.

As mentioned above, the first image may be previously stored by the useror captured by the user in real time. For example, the first image maybe previously stored by the user in the mobile device 102 or captured inreal time by the user using an external camera connected to the mobiledevice 102 or a camera built into the mobile device 102. In anembodiment, the user may also acquire the first image in real timethrough the network. In an embodiment, the first image may also be areduced-resolution image obtained by down-sampling an original imagepreviously stored by the user or an original image captured or acquiredby the user in real time. After down-sampling processing, the amount ofdata processing may be reduced, thereby improving the computationalefficiency of subsequent feature portion determining step and categoryidentification step.

In step S202, based on the first image, the feature portion of theobject is determined.

In an embodiment of the present disclosure, a feature portion of anobject may be one or more key parts that can be used to clearly identifythe category of the object. Objects of a category may have one or morefeature portions. For example, for a flower plant, the feature portionthereof may be its petal portion and/or its flower stem portion.

In an embodiment, the step of determining the feature portion of theobject may include: identifying a preliminary category of the objectbased on the first image; and determining the feature portion of theobject based on the identified preliminary category of the object. In anembodiment, taking the object to be identified as a plant as an example,the step of identifying the preliminary category of the object mayfurther include: acquiring and recording one or more of the locationinformation and season information in the acquisition of the firstimage, excluding impossible object categories according to the one ormore of location information and season information; and in the case ofexcluding impossible object categories, identifying a preliminarycategory of the object. For example, the possibility that the plant is apalm tree may be ruled out according to the location where the usercaptures the first image of a plant in real time in Northeast China,because in China palm trees are usually only distributed in areas southof the Qinling Mountains except Tibet. For example, the possibility thatthe plant is a pear flower may be ruled out according to the fact thatthe season when the user captures a photo of a certain plant in realtime is in winter, because the pear flower usually only blooms inspring.

In an embodiment, a preliminary category of an object may be identifiedthrough a pre-trained first object category identification model basedon the first image. FIG. 3 shows a flowchart of a training method 300 ofan object category identification model according to an embodiment ofthe present disclosure. As shown in FIG. 3 , the training steps of thefirst object category identification model may include: step S301,obtaining a training sample set, each sample in the training sample setis labelled with a corresponding category; step S302, obtaining a testsample set, each sample in the test sample set is labelled with acorresponding category, and the test sample set is different from thetraining sample set; step S303, training the first object categoryidentification model based on the training sample set; step S304,testing the first object category identification model based on the testsample set; step S305, when the test result indicates that theidentification accuracy rate of the first object category identificationmodel is less than a preset accuracy rate, increasing the number ofsamples in the training sample set for retraining; and step S306, whenthe test result indicates that the identification accuracy rate of thefirst object category identification model is greater than or equal tothe preset accuracy rate, completing the training.

For example, a certain number of image samples labelled withcorresponding information are obtained for each plant category, and thenumber of image samples prepared for each plant category may be the sameor different. The corresponding information labelled for each imagesample may include the plant category in the image sample (includingscientific name, alias, category name of botanical classification,etc.). The image sample obtained for each plant category may include, asmany as possible, images of the plants in the category that are capturedat different capturing angles, in different lighting conditions,different weathers (for example, the same plant may have differentappearances in sunny and rainy days), different seasons (for example,the same plant may have different appearances in different seasons), atdifferent times (for example, the same plant may have differentappearances in the morning and at night), in different growthenvironments (for example, the same plant may grow differently indoorsand outdoors), and in different geographical locations (for example, thesame plant may grow differently in different geographic locations). Inthese cases, the corresponding information labelled for each imagesample may further include information such as capturing angle,illumination, weather, season, time, growth environment, or geographiclocation of the image sample.

The image samples subjected to the above labeling process may beclassified into a training sample set for training the first objectcategory identification model and a test sample set for testing thetraining results. Normally the number of samples in the training sampleset is significantly greater than the number of samples in the testsample set. For example, the number of samples in the test sample setmay account for 5% to 20% of the total number of image samples, whilethe number of samples in the corresponding training sample set mayaccount for 80% to 95% of the total image samples. It should beunderstood by those skilled in the art that the number of samples in thetraining sample set and the testing sample set may be adjusted asrequired.

The first object category identification model may be trained using thetraining sample set, and the identification accuracy rate of the trainedfirst object category identification model may be tested using the testsample set. If the identification accuracy rate does not meet therequirements, the number of image samples in the training sample set isincreased, and the updated training sample set is used to retrain thefirst object category identification model until the identificationaccuracy rate of the trained first object category identification modelmeets the requirement. If the identification accuracy rate meets therequirements, the training ends. In an embodiment, whether the trainingcan be ended may be determined based on whether the identificationaccuracy rate is less than the preset accuracy rate. In this way, thetrained first object category identification model whose output accuracyrate meets the requirements may be used for object categoryidentification.

In an embodiment, the first object category identification model may bea deep convolutional neural network (CNN) or a deep residual network(Resnet). Among them, the deep convolutional neural network is a deepfeed-forward neural network, which uses a convolution kernel to scan theplant image, extracts the features to be identified in the plant image,and then identifies the features of the plant to be identified. Inaddition, in the process of identifying plant images, the original plantimages may be directly input into the deep convolutional neural networkmodel without pre-processing the plant images. Compared with otheridentification models, the deep convolutional neural network model hashigher identification accuracy rate and identification efficiency.Compared with the deep convolutional neural network model, the deepresidual network model is added with an identity mapping layer, whichmay avoid the saturation and even decline of accuracy rate caused by theconvolutional neural network as the network depth (the number of layersin the network) increases. The identity mapping function of the identitymapping layer in the residual network model needs to satisfy: the sum ofthe identity mapping function and the input of the residual networkmodel is equal to the output of the residual network model. After theidentity mapping is introduced, the changes of output by the residualnetwork model is more distinguishable, so that the identificationaccuracy rate and identification efficiency of plant physiologicalperiod identification may be significantly improved, thereby improvingthe identification accuracy rate and identification efficiency ofplants.

It should be noted that the concepts of the present disclosure may alsobe implemented using other known or future developed training andidentification models.

Still taking the object to be identified as a plant as an example, in anembodiment, the step of identifying the preliminary category of theobject based on the first image may include: identifying the genusinformation of the object. For example, after the above-mentionedpreliminary category identification processing of the object, it ispossible to only identify the genus information of the object (forexample, peach, cherry or rose, etc.), but the species information ofplant (that is, precise category of plant) cannot be accuratelyidentified. For example, the object is only identified as belonging tothe genus Peach, but it is not possible to determine which peach speciesthat the object belongs to. In this embodiment, the feature portion ofthe object may be determined based on the pre-established correspondencebetween the genus of the plant and a corresponding feature portionthereof. For example, for peach plants, it is possible to make furtherjudgment based on the parts or features of its fruit, petal shape,calyx, overall shape (for example, whether the plant is a tree or ashrub), whether branches are hairy, or whether there are hairs on thefront and back of leaves, so as to further determine the precisecategory of the peach plant. For cherry plants, it is possible to makefurther judgment based on the parts or features of whether the calyx isreflexed, whether the calyx is hairy, the length of the sepal and thecalyx tube, the overall shape of the inflorescence, the bracts, theoverall shape of the leaf, whether both sides of the leaf are hairless,whether the leaf edge is serrated, shape of petal top and shape ofstipule, so as to further determine the precise category of the cherryplant. For rose plants, it is possible to make further judgment based onthe parts or features of whether the calyx is reflexed, whether thecalyx is hairy, the length of the sepal and the calyx tube, the overallshape of the inflorescence, the bracts, the overall shape of the leaf,whether both sides of the leaf are hairless, whether the leaf edge isserrated, shape of petal top, shape of stipule, and whether the flowerstem has thorns or the shape of thorns, so as to further determine theprecise category of the rose plant. Based on the above, the featureportions corresponding to the peach plant that may be pre-establishedinclude: one or more of fruit, petal, calyx, whole plant, branch andleaf parts of the plant, etc.; the feature portions corresponding to thecherry plant that may be pre-established include: one or more of calyx,sepals, calyx tubes, petals, bracts and leaf parts of the plant, etc.;the feature portions corresponding to rose plant that may bepre-established include: one or more of calyx, sepals, calyx tubes,petals, bracts slices, leaves and stem parts of the plant, etc., asshown in Table 1.

TABLE 1 Genus of object-feature portion correspondence table GenusFeature portion Peach fruit, petal, calyx, whole plant, branch, leavesCherry calyx, sepals, calyx tubes, petals, bracts, leaves Rose calyx,sepals, calyx tubes, petals, bracts slices, leaves, stem . . . . . .

In another embodiment, for example, in the case where the genusinformation of the object cannot be identified after preliminaryidentification processing, or in the case where the correspondingrelationship between the genus information of the preliminary identifiedobject and its feature portion has not been established in advance,other methods may be adopted to determine the feature portion of theobject. For example, for a plant object, division may be made based onthe botanical parts, and one or more of its roots, stems, leaves,flowers, and fruit parts may be used as its feature portion. In anembodiment, the flower parts may be further subdivided into multipleparts such as the front part of the petal, the reverse part of thepetal, the side part of the petal, the edge part of the petal, and thepedicel part as the feature portions of the object. In an embodiment,the leaf part may be further subdivided into one or more parts such asthe front part of the leaf, the reverse part of the leaf, the petiolepart, and the edge part of the leaf as the feature portions of theobject.

In step S203, a second image of the feature portion of the object isacquired.

In an embodiment, the step of acquiring the second image of the featureportion of the object may include: providing prompt information to theuser, the prompt information instructing the user to input the secondimage of the feature portion of the object; and receiving the secondimage of the feature portion of the object. In an embodiment, the secondimage has a higher resolution than the first image. For example, if thepreliminary category of the object is identified as peach blossomsaccording to the above-described embodiment (i.e., the plant ispreliminarily identified as belonging to the genus Peach), the systemmay output to the user the prompt information through, for example, aninteractive interface of the mobile device 102, instructing the user toinput the second image of the petal portion (i.e., the feature portioncorresponding to the peach plant) of the object. Then, the system mayreceive the image captured by the user again according to the promptinformation, and use the captured image as the second image of thefeature portion of the object. In an embodiment, the prompt informationmay be provided to the user through one or more of text, graphics andvoice.

In an embodiment, the step of acquiring the second image of the featureportion of the object may include: determining whether the first imageincludes the feature portion of the object; and in the case that thefirst image includes the feature portion of the object: cropping thefirst image to obtain the image of the feature portion as the secondimage, or in the case that the first image is a reduced-resolution imageobtained by down-sampling the third image, cropping the third image toobtain the image of the feature portion as the second image.

Specifically, whether the first image includes a feature portion (e.g.,a petal portion) of an object (e.g., a peach blossom) may be determinedthrough various identification or matching algorithms. In an embodiment,the corresponding region where the feature portion is located in thefirst image may be searched and positioned according to a first objectpart identification model established by pre-training, so as to performsubsequent cropping processing. The first object part identificationmodel may be a deep convolutional neural network (CNN) or a deepresidual network (Resnet), which may be pre-trained based on a largenumber of complete images of different objects and images of variousparts of different objects. In this embodiment, each part of the objectin the first image may be identified and labelled by using the firstobject part identification model. For example, for a flower object, thefirst object part identification model may be used to identify and labeldifferent parts of the flower object respectively, such as leaf parts,petal parts or flower stem parts. Next, the predetermined featureportion of the object (for example, the petal part) is adopted toposition and determine whether the first image includes the featureportion of the object. Still taking the above-mentioned embodiment aboutpeach blossom identification as an example, in the case where it isdetermined that the first image includes the petal part of the object,the first image may be cropped to obtain the image of the petal part asthe second image. In an embodiment, the first image may include petalparts, stem parts, leaf parts and some other picture background elementsof the object, in which case, it is possible to crop and extract thepetal parts only as the second image. In an embodiment, the first imagemay include only the petal parts of the object, but may include multiplepetals, for example, the first image is a peach flower, which includesfive petals in total, in which case, it is possible to crop and extractone of the petals only as the second image. In an embodiment, the firstimage may be a reduced-resolution image obtained by performingdown-sampling processing on the original image captured by the user. Inthis case, the original image corresponding to the first image may alsobe cropped to obtain the image of the petal portion as the second image.Cropping the original image may preserve the original information of theobject to a greater extent, thereby improving the identificationaccuracy. In the case where it is determined that the first image doesnot include the petal part of the object, prompt information may beprovided to the user as described above, and an image captured again bythe user may be received as the second image for subsequent processing,which will not be repeated here.

In an embodiment, the step of acquiring the second image of the featureportion of the object includes: determining whether the first imageincludes a complete image of the object; and in the case where the firstimage includes the complete image of the object: cropping the firstimage to obtain the image of the feature portion as the second image, orin the case that the first image is a reduced-resolution image obtainedby down-sampling the third image, cropping the third image to obtain theimage of the feature portion as the second image.

According to an embodiment of the present disclosure, it may bedetermined whether the first image includes a complete image of theobject through a pre-trained second object part identification model. Inan embodiment, each part of the object in the first image may beidentified and labelled by using the second object part identificationmodel. For example, for a flower-type object, the second object partidentification model may be used to identify and label different partsof the flower-type object respectively, such as leaf parts, petal partsor flower stem parts. Then, whether the first image includes a completeimage of the object may be determined based on a predetermined rule. Inan embodiment, the predetermined rule may be: whether the identified andlabelled parts include all the predetermined parts of the object. Forexample, for a flower-type object, all the pre-determined parts may beleaf parts, petal parts and flower stem parts. In this case, only whenthe identified and labelled parts include leaf parts, petal parts andflower stem parts, it is then determined that the first image includesthe complete image of the object. In another embodiment, thepredetermined rule may be: the number of identified and labelled partsis greater than or equal to a predetermined threshold. For example, alsofor a flower-type object, the predetermined threshold may be 3. In thiscase, only when the number of identified and labelled parts is greaterthan or equal to 3 (for example, including leaf parts, petal parts andflower stem parts), it is then determined that the first image includesthe complete image of the object. It should be understood that whetherthe first image includes a complete image of the object may also bedetermined based on any other predetermined rule. The second object partidentification model may be a deep convolutional neural network (CNN) ora deep residual network (Resnet). In an embodiment, the second objectpart identification model may be trained based on a large number ofcomplete images of different objects and images of various parts ofdifferent objects. The second object part identification model and thefirst object part identification model may be the same model ordifferent models. Similar to the above description, in the case where itis determined that the first image includes the complete image of theobject, the first image or the original image corresponding to the firstimage may be cropped to obtain the image of the feature portion as thesecond image; in the case where it is determined that the first imagedoes not include the complete image of the object, prompt informationmay be provided to the user as described above, and an image capturedagain by the user may be received as the second image for subsequentprocessing, which will not be repeated here.

Finally, in step S204, based on the second image, the object category ofthe object is identified.

In an embodiment, a pre-trained second object category identificationmodel may be used to identify the object category of the object. Thesecond object category identification model and the first objectcategory identification model may be the same model or different models.For example, the second object category identification model may be adeep convolutional neural network (CNN) or a deep residual network(Resnet). In an embodiment, the second object category identificationmodel may also be trained using the training method 300 shown in FIG. 3. Compared with the above-mentioned preliminary classification ofobjects based on the first image identification, in step S204, since thesecond image may have an image resolution greater than or equal to thefirst image, and may include the feature portion of the object morespecifically than the first image, the identification result in stepS204 may be made more accurate. In an embodiment, since objects of thesame category might grow differently in different geographic locations(e.g., grown in the south or north), in different seasons (e.g., inspring or in fall), at different times (e.g., in the morning orevening), under different weathers (e.g., sunny or rainy days) and atdifferent capturing angles, in addition to identifying the categoryinformation of the object, identifying the object category of the objectmay further include identifying one or more of the location information,season information, time information, weather information, and capturingangle information associated with the object. For example, the categoryof the plant in the image may be identified as sunflower according tothe image captured by the user, and the plant may also be identified asa sunflower on a sunny day based on its upright and slightly tiltedflower disc (the sunflower flower disc on a cloudy day is normallyslightly lower). For example, the category of plant in the image may beidentified as ginkgo tree according to the image captured by the user,and the plant may also be identified based on the color of its leavesthat the ginkgo tree is currently captured in autumn (ginkgo leaves turnyellow in autumn).

In addition, the original image captured by the user, the first imageafter the original image is compressed and down-sampled, or the obtainedsecond image of the feature portion of the object may be stored in thesample library corresponding to the object category of the object, andphysiological cycle information and appearance information correspondingto one or more of location information, season information, timeinformation, weather information, and capturing angle information of theobject are recorded.

FIG. 4 shows a flowchart of an object identification method 400according to another embodiment of the present disclosure.

According to the embodiment shown in FIG. 4 , first, at 401, an originalimage of the object may be acquired. At 402, the original image of theobject may be down-sampled to obtain a first image with reducedresolution. Then, at 403, it may be determined whether the first imageincludes the complete object or a portion of the object based on thepre-trained second object part identification model described above. Ifit is determined that the first image contains the complete object, theprocess proceeds to 404. At 404, a second image corresponding to afeature portion of the object may be cropped and extracted from thefirst image containing the complete object or a corresponding originalimage. If it is determined at 403 that the first image contains only aportion of the object, the process proceeds to 405. At 405, it may befurther determined whether the first image containing only a part of theobject corresponds to the feature portion of the object, if so, theprocess proceeds to 404 to perform corresponding processing; if not, theprocess proceeds to 406. At 406, prompt information may be output to theuser, the prompt information instructing the user to input a secondimage of the feature portion of the object. Next, at 407, a second imagemay be received based on the user's input. After acquiring the secondimage corresponding to the feature portion of the object at 404 or 407,the process may proceed to 408. Finally, at 408, an object category ofthe object may be identified based on the second image, e.g., as may beimplemented by a pre-trained object category identification model asdescribed above.

FIG. 5 shows a schematic view of an object identification apparatus 500according to an embodiment of the present disclosure.

The object identification apparatus 500 according to an embodiment ofthe present disclosure may include: an image acquisition module 501, afeature portion determining module 502, and an object categoryidentification module 503. The image acquisition module 501 isconfigured to acquire a first image of at least a portion of the object,and to acquire a second image of a feature portion of the object; thefeature portion determining module 502 is configured to determine afeature portion of the object based on the first image; and the objectcategory identification module 503 is configured to identify the objectcategory of the object based on the second image of the feature portionof the object.

In an embodiment, the step of determining the feature portion of theobject may include: identifying a preliminary category of the objectbased on the first image; and determining the feature portion of theobject based on the identified preliminary category of the object.

In an embodiment, the step of acquiring the second image of the featureportion of the object may include: providing prompt information to theuser, the prompt information instructing the user to input the secondimage of the feature portion of the object; and receiving the secondimage of the feature portion of the object.

In an embodiment, the step of obtaining the second image of the featureportion of the object may include: determining whether the first imageincludes the feature portion of the object; and in the case that thefirst image includes the feature portion of the object, cropping thefirst image to obtain the image of the feature portion as the secondimage, or in the case that the first image is a reduced-resolution imageobtained by down-sampling the third image, cropping the third image toobtain the image of the feature portion as the second image.

In an embodiment, the step of acquiring the second image of the featureportion of the object may include: determining whether the first imageincludes the complete image of the object; and in the case that thefirst image includes the complete image of the object, cropping thefirst image to obtain the image of the feature portion as the secondimage, or in the case that the first image is a reduced-resolution imageobtained by down-sampling the third image, cropping the third image toobtain the image of the feature portion as the second image.

FIG. 6 shows a schematic view of an object identification device 600according to an embodiment of the present disclosure.

As shown in FIG. 6 , the object identification device 600 according toan embodiment of the present disclosure may include: an imageacquisition unit 601, a processor 602 and a memory 603.

The image acquisition unit 601 may be any image receiving unit capableof acquiring various forms of input images. The image acquisition unit601 may acquire previously stored images, images captured by the user inreal time, or may acquire images in real time directly through thenetwork.

The processor 602 may perform various actions and processes according toprograms stored in the memory 603. Specifically, the processor 602 maybe an integrated circuit chip with signal processing capability. Theaforementioned processors may be general-purpose processors, digitalsignal processors (DSPs), application specific integrated circuits(ASICs), off-the-shelf programmable gate arrays (FPGAs) or otherprogrammable logic devices, discrete gate or transistor logic devices,discrete hardware components. Various methods, steps and logic blockdiagrams disclosed in the embodiments of the present disclosure may beimplemented or executed. The general-purpose processor may be amicroprocessor or the processor may also be any conventional processor,etc., and may be an X86 architecture or an ARM architecture, or thelike.

The memory 603 stores executable instruction codes, and the instructioncodes implement the object identification method 200 or the objectidentification method 400 described above when being executed by theprocessor 602. The memory 603 may be volatile memory or nonvolatilememory, or may include both volatile memory and nonvolatile memory.Non-volatile memory may be read-only memory (ROM), programmableread-only memory (PROM), erasable programmable read-only memory (EPROM),electrically erasable programmable read-only memory (EEPROM), or flashmemory. Volatile memory may be random access memory (RAM), which servesas an external cache. By way of example and not limitation, many formsof RAM are available, such as static random access memory (SRAM),dynamic random access memory (DRAM), synchronous dynamic random accessmemory (SDRAM), double data rate synchronous dynamic random accessmemory (DDRSDRAM), enhanced synchronous dynamic random access memory(ESDRAM), synchronous link dynamic random access memory (SLDRAM), anddirect memory bus random access memory (DRRAM). It should be noted thatthe memory of the methods described herein is intended to include, butnot be limited to, these and any other suitable types of memory.

The present disclosure further provides a computer-readable storagemedium having computer-executable instructions stored thereon, thecomputer-executable instructions implement the object identificationmethod 200 or the object identification method 400 described above whenbeing executed by a processor. Similarly, the computer-readable storagemedium in embodiments of the present disclosure may be volatile memoryor non-volatile memory, or may include both volatile memory andnon-volatile memory. It should be noted that computer-readable storagemedium described herein are intended to include, but not be limited to,these and any other suitable types of memory.

The method, apparatus and device for object identification provided bythe present disclosure are simple and easy to operate and mayeffectively improve the accuracy of object identification by acquiringthe feature portion of the object and identifying the object categorybased on the feature portion.

It should be noted that the flowcharts and block diagrams in theaccompanying drawings illustrate the architecture, functionality, andoperation of possible implementations of systems, methods and computerprogram products according to various embodiments of the presentdisclosure. In this regard, each block in the flowchart or blockdiagrams may represent a module, segment, or portion of code thatcontains one or more executable instructions for implementing thespecified logic function. It should also be noted that, in somealternative implementations, the functions labelled in the blocks mayoccur differently from the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in a reverseorder, depending upon the functionality involved. It should also benoted that each block of the block diagrams and/or flowchartillustrations, and combinations of blocks in the block diagrams and/orflowchart illustrations, may be implemented in specific hardware-basedsystems that perform the specified functions or operations, or may beimplemented in a combination of special hardware and computerinstructions.

In general, the various exemplary embodiments of the present disclosuremay be implemented in hardware or special-purpose circuits, software,firmware, logic, or any combination thereof. Certain aspects may beimplemented in hardware, while other aspects may be implemented infirmware or software that may be executed by a controller,microprocessor or other computing device. While aspects of theembodiments of the present disclosure are illustrated or described asblock diagrams, flowcharts, or represented by some other graphics, it isto be understood that the blocks, apparatus, systems, techniques, ormethods described herein may be taken as non-limiting examplesimplemented in hardware, software, firmware, special-purpose circuits orlogic, general-purpose hardware or controllers or other computingdevices, or some combination thereof.

The exemplary embodiments of the present disclosure described in detailabove are illustrative only and not restrictive. It should be understoodby those skilled in the art that various modifications and combinationsof these embodiments or features thereof may be made without departingfrom the principles and spirit of the disclosure, and such modificationsshould fall within the scope of the disclosure.

1. An object identification method, comprising: acquiring a first imageof at least a portion of an object; determining a feature portion of theobject based on the first image; acquiring a second image of the featureportion of the object; and identifying an object category of the objectbased on the second image.
 2. The method according to claim 1, whereinthe step of determining the feature portion of the object comprises:identifying a preliminary category of the object based on the firstimage; and determining the feature portion of the object based on thepreliminary category of the object, which is identified.
 3. The methodaccording to claim 1, wherein the step of acquiring the second image ofthe feature portion of the object comprises: providing promptinformation to a user, the prompt information instructing the user toinput the second image of the feature portion of the object; andreceiving the second image of the feature portion of the object.
 4. Themethod according to any one of claim 1, wherein the step of acquiringthe second image of the feature portion of the object comprises:determining whether the first image comprises the feature portion of theobject; and when the first image comprises the feature portion of theobject: cropping the first image to obtain an image of the featureportion as the second image, or in the case where the first image is areduced-resolution image after down-sampling a third image, cropping thethird image to obtain the image of the feature portion as the secondimage.
 5. The method according to claim 4, wherein the step ofdetermining whether the first image comprises the feature portion of theobject comprises: identifying and labeling each part of the object inthe first image through a pre-trained first object part identificationmodel; and determining whether the first image comprises the featureportion of the object based on an identification and labeling result. 6.The method according to any one of claim 1, wherein the step ofacquiring the second image of the feature portion of the objectcomprises: determining whether the first image comprises a completeimage of the object; and in the case that the first image comprises thecomplete image of the object: cropping the first image to obtain animage of the feature portion as the second image, or in the case wherethe first image is a reduced-resolution image after down-sampling athird image, cropping the third image to obtain the image of the featureportion as the second image.
 7. The method according to claim 2, whereinthe object is a plant, and a resolution of the second image is higherthan a resolution of the first image, wherein the step of identifyingthe preliminary category of the object comprises: acquiring andrecording location information and/or season information in acquisitionof the first image, based on the one or more object categories thatexclude impossible in the location information and the seasoninformation; and in the case of excluding the object categories that areimpossible, identifying the preliminary category of the object.
 8. Themethod according to claim 7, wherein the step of identifying the objectcategory of the object comprises identifying one or more information ofcategory information, the location information, the season information,time information, weather information, and capturing angle informationassociated with the object.
 9. The method according to claim 8, whereinthe first image and/or the second image are stored in a sample librarycorresponding to the object category of the object, and physiologicalcycle information and appearance information corresponding to the one ormore information the location information, the season information, thetime information, the weather information, and the capturing angleinformation are recorded.
 10. The method according to claim 3, whereinthe step of providing the prompt information to the user comprises:providing the prompt information to the user through text, graphics,and/or voice.
 11. The method according to claim 2, wherein thepreliminary category of the object is identified based on a first objectcategory identification model, which is pre-trained; the object categoryof the object is identified based on a second object categoryidentification model, which is pre-trained, wherein the first objectcategory identification model and the second object categoryidentification model are the same or different; the first objectcategory identification model comprises a deep convolutional neuralnetwork or a deep residual network, and the second object categoryidentification model comprises the deep convolutional neural network orthe deep residual network.
 12. The method according to claim 11, whereina step of pre-training the first object category identification modelcomprises: acquiring a training sample set, wherein each of samples inthe training sample set is labeled with a corresponding category;acquiring a test sample set, wherein each of samples in the test sampleset is labeled with a corresponding category, and the test sample set isdifferent from the training sample set; training the first objectcategory identification model based on the training sample set; testingthe first object category identification model based on the test sampleset; when a test result indicates that an identification accuracy rateof the first object category identification model is less than a presetaccuracy rate, increasing the number of the samples in the trainingsample set for retraining; and when the test result indicates that theidentification accuracy rate of the first object category identificationmodel is greater than or equal to the preset accuracy rate, completingthe training.
 13. The method according to claim 11, wherein a step ofpre-training the second object category identification model comprises:acquiring a training sample set, wherein each of samples in the trainingsample set is labeled with a corresponding category; acquiring a testsample set, wherein each of samples in the test sample set is labeledwith a corresponding category, and the test sample set is different fromthe training sample set; training the second object categoryidentification model based on the training sample set; testing thesecond object category identification model based on the test sampleset; when a test result indicates that an identification accuracy rateof the second object category identification model is less than a presetaccuracy rate, increasing the number of the samples in the trainingsample set for retraining; and when the test result indicates that theidentification accuracy rate of the second object categoryidentification model is greater than or equal to the preset accuracyrate, completing the training.
 14. An object identification apparatus,comprising: an image acquisition module configured to acquire a firstimage of at least a portion of an object, and to acquire a second imageof a feature portion of the object; a feature portion determining moduleconfigured to determine the feature portion of the object based on thefirst image; and an object category identification module configured toidentify an object category of the object based on the second image ofthe feature portion of the object.
 15. The apparatus according to claim14, wherein the step of determining the feature portion of the objectcomprises: identifying a preliminary category of the object based on thefirst image; and determining the feature portion of the object based onthe preliminary category of the object, which is identified.
 16. Theapparatus according to claim 14, wherein the step of acquiring thesecond image of the feature portion of the object comprises: providingprompt information to a user, the prompt information instructing theuser to input the second image of the feature portion of the object; andreceiving the second image of the feature portion of the object.
 17. Theapparatus according to any one of claim 14, wherein the step ofacquiring the second image of the feature portion of the objectcomprises: determining whether the first image comprises the featureportion of the object; and in the case that the first image comprisesthe feature portion of the object: cropping the first image to obtain animage of the feature portion as the second image, or in the case wherethe first image is a reduced-resolution image after down-sampling athird image, cropping the third image to obtain the image of the featureportion as the second image.
 18. The apparatus according to any one ofclaims 14 16 claim 14, wherein the step of acquiring the second image ofthe feature portion of the object comprises: determining whether thefirst image comprises a complete image of the object; and in the casethat the first image comprises the complete image of the object:cropping the first image to obtain an image of the feature portion asthe second image, or in the case where the first image is areduced-resolution image after down-sampling a third image, cropping thethird image to obtain the image of the feature portion as the secondimage.
 19. An object identification device, comprising: an imageacquisition unit for acquiring an input image; a processor; and a memoryconfigured to store a series of computer-executable instructions and aseries of computer-accessible data associated with the series ofcomputer-executable instructions, wherein the series ofcomputer-executable instructions enable the processor to perform themethod claimed in claim 1 when being executed by the processor.